{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Vectorspace Orthonormalization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A function that converts a set of linearly independent vectors to a set of orthonormal basis vectors."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> from mlxtend.math import vectorspace_orthonormalization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `vectorspace_orthonormalization` converts a set linearly independent vectors to a set of orthonormal basis vectors using the Gram-Schmidt process [1]. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References\n",
    "\n",
    "- [1] https://en.wikipedia.org/wiki/Gram–Schmidt_process"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 1 - Convert a set of vector to an orthonormal basis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that to convert a set of linearly independent vectors into a set of orthonormal basis vectors, the `vectorspace_orthonormalization` function expects the vectors to be arranged as columns of a matrix (here: NumPy array). Please keep in mind that the `vectorspace_orthonormalization` function also works for non-linearly independent vector sets; however, the resulting vectorset won't be orthonormal as a result. An easy way to check whether all vectors in the input set are linearly independent is to use the `numpy.linalg.det` (determinant) function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input vectors are linearly independent\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[ 0.40824829, -0.1814885 ,  0.04982278,  0.89325973],\n",
       "       [ 0.        ,  0.1088931 ,  0.99349591, -0.03328918],\n",
       "       [ 0.81649658,  0.50816781, -0.06462163, -0.26631346],\n",
       "       [ 0.40824829, -0.83484711,  0.07942048, -0.36063281]])"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "from mlxtend.math import vectorspace_orthonormalization\n",
    "    \n",
    "a = np.array([[2,   0,   4,  12],\n",
    "              [0,   2,  16,   4],\n",
    "              [4,  16,   6,   2],\n",
    "              [2, -12,   4,   6]])\n",
    "\n",
    "\n",
    "s = ''\n",
    "if np.linalg.det(a) == 0.0:\n",
    "    s = ' not'\n",
    "print('Input vectors are%s linearly independent' % s)\n",
    "\n",
    "\n",
    "vectorspace_orthonormalization(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that scaling the inputs equally by a factor should leave the results unchanged:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.40824829, -0.1814885 ,  0.04982278,  0.89325973],\n",
       "       [ 0.        ,  0.1088931 ,  0.99349591, -0.03328918],\n",
       "       [ 0.81649658,  0.50816781, -0.06462163, -0.26631346],\n",
       "       [ 0.40824829, -0.83484711,  0.07942048, -0.36063281]])"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vectorspace_orthonormalization(a/2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, in case of linear dependence (the second column is a linear combination of the first column in the example below), the vector elements of one of the dependent vectors will become zero. (For a pair of linear dependent vectors, the one with the larger column index will be the one that's zero-ed.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.40824829,  0.        ,  0.04155858,  0.82364839],\n",
       "       [ 0.        ,  0.        ,  0.99740596, -0.06501108],\n",
       "       [ 0.81649658,  0.        , -0.04155858, -0.52008861],\n",
       "       [ 0.40824829,  0.        ,  0.04155858,  0.21652883]])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a[:, 1] = a[:, 0] * 2\n",
    "vectorspace_orthonormalization(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "## vectorspace_orthonormalization\n",
      "\n",
      "*vectorspace_orthonormalization(ary, eps=1e-13)*\n",
      "\n",
      "Transforms a set of column vectors to a orthonormal basis.\n",
      "\n",
      "Given a set of linearly independent vectors, this functions converts such\n",
      "column vectors, arranged in a matrix, into orthonormal basis\n",
      "vectors.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `ary` : array-like, shape=[num_vectors, num_vectors]\n",
      "\n",
      "    A set of vectors (arranged as columns in a matrix)\n",
      "\n",
      "\n",
      "- `eps` : float (default: 1e-13)\n",
      "\n",
      "    A small tolerance value to determine whether\n",
      "    the vector norm is zero or not.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `arr` : array-like, shape=[num_vectors, num_vectors]\n",
      "\n",
      "    An orthonormal set of vectors (arranged as columns)\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open('../../api_modules/mlxtend.math/vectorspace_orthonormalization.md', 'r') as f:\n",
    "    print(f.read())"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
